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About this MHEC Research Brief Series 

This research brief is drawn from specific topics examined in the forthcoming MHEC report, 
Institutional Practices Conducive to Student Success: An Overview of Theory and Research. 

Correspondence concerning this brief should be sent to Aaron Horn, Associate Director for 
Policy Research, aaronh@mhec.org. 
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Colleges and universities are increasingly adopting various software solutions to raise degree 
completion rates and lower costs (Ferguson, 201 2; Vendituoli, 201 4; Yanosky, 201 4). Student 
success software, also known as Integrated Planning and Advising Services (IPAS), appears to be in 
high demand among both students and faculty (Dahlstrom & Bichsel, 2014; Dahlstrom & Brooks, 

201 4). For example, in a recent survey of students at 1 85 postsecondary institutions, over 80% of 
respondents indicated at least moderate interest in using software for guiding course selection, 
monitoring and improving performance, and identifying relevant resources (Dahlstrom & Bichsel, 
2014). 

This brief provides an overview of student success software and summarizes findings from the 
nascent body of student outcomes research. Student success software can be generally classified 
as one of three types: academic planning systems, task engagement systems, and early alert 
systems (see Table 1). Academic planning software can assist students in creating a degree plan and 
monitoring degree progress. Task engagement software can provide automated behavioral cues, 
reminders, and positive reinforcement that help students to complete coursework and comply with 
administrative deadlines. Early warning systems are designed to collect and utilize student data to 
alert faculty and staff of students in need of assistance. This brief concludes with the identification of 
several campus practices that may promote the successful implementation of new software solutions. 


Tablel . Examples of Software Products for Promoting Student Success 


Vendor or Institution 

Product 

Academic Planning Systems 


Civitas Learning 

Degree Map 

CollegeSource 

u. achieve & u. direct 

Desire2Learn 

Brightspace Degree Compass 

Edunav 

Edunav 

Hobsons 

AgileGrad 

Oracle 

PeopleSoft Campus Solutions 

Redrock 

TutorTrac & AdvisorTrac 

Starfish Retention Solutions 

Starfish ADVISING 

Student Success Plan 

Student Success Plan 

Valencia Community College 

LifeMap 

Virginia Community College System 

Virginia Education Wizard 
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Tablel . Examples of Software Products for Promoting Student Success (continued) 


Vendor or Institution 

Product 



Task Engagement Systems 


Beyond 12 

MyCoach 

GradGears 

GradGuru 

Persistence Plus 

Persistence Plus 



Early Alert Systems 


Blackboard 

Blackboard Analytics Suite 

Campus Labs 

Beacon 

Purdue University 

Course Signals 

Education Advisory Board 

Student Success Collaborative 

Ellucian 

Colleague Retention Alert 

Ellucian 

Banner Student Retention Performance 

EMAS 

Retention Pro 

Hobsons 

Retain 

Jenzabar 

Jenzabar JX 

EBI 

Map Works 

Pharos 

Pharos 360 

QuScient 

ProRetention 

SARS 

SARS Anywhere 

SmartEvals 

DropGuard 

Starfish Retention Solutions 

Starfish Early Alert 


Academic Planning Systems 

Academic planning software has been developed to assist students in creating a degree plan 
and monitoring degree progress. Degree plans specify career objectives and curricular roadmaps; 
some programs also recommend courses based on degree relevance and the likelihood of success. 
For instance, Austin Community College incorporated a Civitas Learning program called Degree 
Map into the advising process, which provided both the student and advisor a visual depiction of 
the student's current program inclusive of all credits completed toward the degree. Degree Map 
also showed "comparisons of degree plan options, the impact of switching degree plans, and the 
identification of additional certifications for which the student may (nearly) be eligible to apply" 
(Brooks, 2014, p.7). In another example, Degree Compass at Austin Peay State University utilizes 
predictive analytics with transcript data to make personalized course recommendations that attempt 
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to match student abilities and program requirements. "From the courses that apply directly to the 
student's program of study, the system selects those courses that fit best with the sequence of 
courses in their degree and are the most central to the university curriculum as a whole. That ranking 
is then overlaid with a model that predicts which courses the student will achieve their best grades" 
(Whitten, Sanders, & Stewart, 2013, p. 39). Academic planning software should not replace advisors, 
though, as students prefer a mix of self-service tools and face-to-face interaction with advisors 
(Ya nosky, 2014). 

Heretofore, evaluations of academic planning systems have only been descriptive in nature 
(Herndon, 201 1; Shugart & Romano, 2008; Whitten, Sanders, & Stewart, 2013). In his examination 
of the Virginia Education Wizard, a statewide online portal for career and college planning, 

Herndon (201 1) reported that 29% of Wizard users at community colleges had received financial 
aid, compared to 26% of non-users. In addition, 46% of Wizard users earned a GPA of 3.0 or higher, 
compared to 38% of non-users. However, this study did not control for pre-existing characteristics of 
users and non-users. 

Task Engagement Software 

Task engagement software provides automated behavioral cues, reminders, and positive 
reinforcement that help students to complete coursework and comply with administrative deadlines 
(e.g., financial aid). Several task engagement software applications have been designed for 
mobile phones, though only Persistence Plus has been the focus of research. Persistence Plus uses 
demographic and behavioral data to send daily "nudges" to students that are intended to promote 
task completion, motivation, and resource utilization (Carmean & Frankfort, 2013). For example, 
students may receive a task completion nudge before an exam - "Several people have tests coming 
up! Where are you going to study this weekend?" - which is followed by a blank text box and a 
submit button for the student to respond. Persistence Plus also provides students with "LifeBits," 
which are narrative accounts of students with similar backgrounds (e.g., ethnicity, gender, first- 
generation) who have successfully overcome obstacles and completed college. 

Two studies of Persistence Plus have been conducted with simple comparisons of participants and 
non-participants. The University of Washington Tacoma (UWT) used Persistence Plus in two online 
courses, mathematics and economics. An evaluation of student outcomes revealed that course 
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completion rates were 12 percentage points higher among Persistence Plus students in math and 19 
percentage points higher among Persistence Plus students in economics, relative to students who 
did not use the software (Carmean & Frankfort, 201 3). Another study was conducted at Middlesex 
Community College, where Maslin, Frankfort, and Jaques-Leslie (201 4) found that the fall to spring 
retention rate was seven percentage points higher among 300 Persistence Plus students, relative 
to the retention rate of non-participants. Additional research is needed to determine whether these 
differences can be attributed to task engagement software. 

Early Warning Systems 

Early warning systems are designed to collect and utilize student data to alert faculty and staff 
of students in need of assistance (Bruce et a I . , 201 1). Over 90% of four-year institutions and 70% of 
community colleges have adopted some type of early warning system (Barefoot, Griffin, & Koch, 

201 2; College Board, 201 2). Although the general goal of early warning systems is universal, systems 
differ in their ability to monitor the effects of student interventions and link with commercial student 
surveys, learning management systems, and student information systems (Hanover Research, 

2013). Moreover, whereas some early warning systems rely on faculty to identify students who 
are struggling with coursework, other systems use predictive analytics to automatically flag at-risk 
students (MacFayden & Dawson, 201 0). Table 2 summarizes other prominent variations in how these 
systems are used at four-year institutions. For example, whereas over half of institutions targeted all 
sophomore students, only 35% of institutions used early warning systems for all first-year students. 
Absenteeism was the most common indicator used to trigger a warning; indicators of academic 
effort and grades below a "C" were less common. Interestingly, 30% of institutions did not monitor 
students continuously, and only 39% of institutions required students to obtain assistance when 
flagged. 

Despite the widespread adoption of early warning systems, few studies on system effectiveness 
have been published. In fact, in their national survey of four-year institutions, Barefoot, Griffin, 
and Koch (201 2) found that 28% of institutions had not evaluated the effectiveness of their early 
alert systems. Less than half of campuses surveyed detected positive effects: only 40% reported 
improvements in persistence or graduation rates, and only 36% reported improvements in academic 
performance (Barefoot et al., 2012). 
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Particular institutional evaluations suggest that early warning systems are perceived as useful 
and may be associated with student success (Arnold & Pistilli, 2012; Campbell, DeBlois, & Oblinger, 
2007; Faulconer et al. ( 2014; Picciano, 2012). Faulconer et al. (2014) examined the implementation 
of the Starfish Retention system at East Carolina University and found that over 90% of students who 
had received a kudos flag reported that it was motivational; and 85% of students who had received 
an academic warning reported taking some action. Rio Salado Community College determined 
that their early warning system had assisted staff in identifying the challenges that students faced, 
adding necessary context to conversations about institutional improvement (Picciano, 201 2). Finally, 
at Sinclair Community College, new students who had completed an individual learning plan that 
provided alerts to advisors had a retention rate of 93%, compared to a retention rate of 65% among 
non-participants (Campbell, DeBlois, & Oblinger, 2007). However, it remains unclear whether these 
differences are attributable to the adoption of an early warning system. 


Table 2. Characteristics of Early Warning Systems at Four- Year Institutions. 


Characteristics 

Percent of Institutions 

Targeted Students 


All first-year students 

35 

Some first-year students (e.g., student athletes) 

27 

All sophomores 

58 

Some sophomores (e.g.,. on academic probation) 

27 

All transfer students 

56 

Some transfer students 

21 



Behaviors that Trigger Warning 


Absenteeism 

90 

Failing grades 

84 

Behavioral problems 

71 

Grades below a "C" 

65 

Low participation or effort 

60 



Contact Features 


Students are contacted 

91 

Students are informed about how to seek assistance 

85 

Students are monitored continuously 

70 

Students are contacted face-to-face 

61 




6 Student Success Software 


Table 2. Characteristics of Early Warning Systems at Four- Year Institutions (continued) 


Characteristics 

Percent of Institutions 

Contact Features (continued) 


Students are required to obtain assistance 

39 

Students' families are notified after a waiver of privacy rights 

17 



Staff Involvement 


Academic advisors 

89 

Faculty 

89 

Academic support personnel 

83 

Athletic department staff 

74 

Counseling staff 

53 

Residence hall staff 

52 


Source. Barefoot, B. O., Griffin, B. Q., & Koch, A. K. (2012). Enhancing student success and retention throughout 
undergraduate education. 


Successful Implementation 

A recent set of case studies identified several practices that characterize successful 
implementation of new student success software (Karp & Fletcher, 201 5; see also Brooks, 201 4). 

First, successful project teams fulfilled three types of roles: content masters, influencers, and decision 
makers. Content masters such as IT staff and advisors provide the necessary technical knowledge; 
influencers are trusted faculty and administrators who develop support for the project, particularly 
by ensuring that members of the campus community understand the benefits of product adoption; 
and decision makers are those who "have authority to move the project forward or who can cast a 
vote for a constituency of the community" (Karp & Fletcher, 2015, p. 6). Trust in the project team 
was cultivated by maintaining openness and transparency at all stages. Moreover, Brooks (2014) 
emphasized the importance of building campus buy-in through the support of senior leadership - 
including the president, vice president, and provost - as well as the college business office, faculty, 
staff, and students. 

Second, although new software should add value to student and staff activities, product adoption 
may inadvertently result in the loss of prior system capabilities and frustrate end-users. Successful 
implementation was thus facilitated by testing product options to confirm the presence of desired 
improvements. A pilot test may also help identify needs for training or resources, such as templates 
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for writing emails when contacting flagged students (Arnold & Pisti Ili, 2012). 

Third, in addition to meeting the technical requirements of a particular software product, 
successful colleges considered whether product adoption would require any changes in institutional 
policies and procedures. Degree planning software, for instance, may require the specification of 
program requirements, course schedules, and course transfer protocols. Implementation may be 
delayed or disrupted if product integration challenges are not addressed upfront. 

Fourth, software implementation is rarely a smooth process absent any technical glitches. 
Although campus IT staff can address some integration issues, other problems may require direct 
support from the vendor. However, software vendors in the implementation study differed greatly 
in their technical support services. Some vendors were unresponsive to support requests, whereas 
others provided representatives who joined project teams and attended meetings. 

Finally, successful colleges anticipated and managed the hidden costs of post-launch 
implementation. For example, end-users in the implementation study "needed time post-rollout to 
experiment with the system, report problems, and figure out how best to leverage the tool" (Karp 
& Fletcher, 2015, p. 8). Time should also be allocated to evaluate the impact of software adoption 
on program goals (Brooks, 201 4). A Total Cost of Ownership analysis will certainly reveal other 
hidden costs of software adoption (e.g., additional personnel, facilities, system upgrades), which are 
thought to be five to ten times greater than the software purchase price over the course of five years 
(Schmidt, 201 5). 

Recommended Practices 

Software products are widely perceived as vital to strategies for increasing student success rates 
(Yanosky, 201 4), but contentions about their actual impact on degree completion or academic 
achievement have not yet undergone rigorous scientific study. Nonetheless, the favorable results of 
descriptive studies suggest that further experimentation with student success software is warranted. 

• The impact of any particular software product will partly depend upon whether software adoption 
advances the college's student success strategy. Colleges should conduct a needs assessment to 
identify software products that address actual problems in academic planning, task engagement, 
and feedback. 

• Project team members should fulfill three types of roles: content masters, influencers, and 
decision makers. 
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• Promote campus buy-in through the support and participation of senior leadership, stakeholders, 
and possible end-users, such as the president, vice president, provost, chief information officer, 
director of institutional research, the college business office, faculty, staff, and students. 

• Cultivate trust in the project team by maintaining openness and transparency at all stages. 

• Pilot test product options to confirm the presence of desired improvements and identify needs 
for training and resources. 

• Consider whether product adoption will require any changes in institutional policies and 
procedures. Address product integration problems before implementation. 

• Ensure that software vendors are willing to provide the level of technical support needed. 

• Budget time and resources for the total cost of software adoption, including post-launch 
implementation, product refinement, and outcomes evaluation. A Total Cost of Ownership 
analysis should be conducted to identify the hidden costs of software adoption (e.g., personnel, 
facilities, system upgrades). 
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